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* DURING THIS PERIOD FROM SEPTEMBER 1 THROUGH NOVEMBER 30, 
1966, 35 FIELD CONSULTANTS COMPLETED ANALYSIS OF THEIR 
INDIVIDUAL TRADE AND TECHNICAL CURRICULUMS. THESE ANALYSES 
WERE DEVELOPED INTO AN OUTLINE TO SERVE AS A GUIDE FOR 
DEVELOPING ACHIEVEMENT TESTS. THE FINAL OUTLINE WAS DIVIDED 
INTO AS MANY DIFFERENT AREAS AS THE CONSULTANTS FELT 
NECESSARY TO REPRESENT INDEPENDENT AREAS OF INSTRUCTION. E/JCH 
INDEPENDENT AREA WAS THEN SUBDIVIDED TO DETAIL THE VARIOUS 
ELEMENTS OF THE CURRICULUM. THE BREAKDOWN WILL BE USED TO 
INSURE A TEST ITEM POOL REPRESENTATIVE OF ALL AREAS OF THE 
CURRICULUM. REFERENCE TESTS WITH KNOWN RELIABILITY AND 
VAI..IDITY ARE BEING EVALUATED TO DETERMINE WHICH WOULD BE MOST 
APPROPRIATE IN THE INITIAL TEST BATTERY AS A BASIS FOR 
COMPARING PROJECT TEST RESULTS. TWELVE NORTH CAROLINA 
TECHNICAL INSTITUTES PARTICIPATED IN THE INITIAL PHASE OF THE 
PROJECT, AND 10 ADDITIONAL INSTITUTIONS HAVE AGREED TO 
PARTICIPATE IN ADMINISTERING THE PRELIMINARY FORMS OF THE 
TEST. METHODS AND TECHNIQUES OF EVALUATING THE PRELIMINARY 
FORMS AND THE PROBLEMS OF DATA STORAGE AND RETRIEVAL HAVE 
BEEN STUDIED. EXPERIMENTAL STUDIES OF KINESTHETIC SENSITIVITY 
WERE CONDUCTED AND APPARATUS BUILT TO INVESTIGATE THIS AREA 
IN MORE DETAIL. AUDITORY RESPONSE AND ITS RELATIONSHIP TO 
ACHIEVEMENT IN SEVERAL OF THE VOCATIONAL AREAS WERE 
INVESTIGATED, AND HIGH-FIDELITY RECORDING EQUIPMENT HAS BEEN 
REQUISITIONED TO RECORD NORMAL AND ABNORMAL SOUNDS FOR 
DIAGNOSTIC WORK IN AUTO MECHANICS, MACHINE SHOP, AND RADIO-TV 
REPAIR. TWO PAPERS ARE GIVEN IN THE APPENDIXES — "USE OF 
OBJECTIVES IN ITEM CONSTRUCTION" AND "AUDITORY DIAGNOSIS 
STUDY." CHC) 
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Since September 1, all additional instructors needed as field 
consultants have been contacted and placed under contract. Each of these 
instructors was visited and the objectives and operating procedures of the 
research project were explained in detail. At this time, the total number 
of instructors working as field consultants number approximately 35 for an 
average of five field consultants per curriculum. 

Etch instructor working on the project w^s asked to do a complete 
analysis of that portion of the curriculum considered to be the trade or 
technical specialty. This analysis was completed by mid-October. During 
the last three weekends in October, meetings were held with the field 
consultants in each of the seven technical or trade curricula included in 
the research project. The purpose of these meetings was first to arrive at 
a consensus concerning the curriculum analysis, secondly to plan ways and 
means of developing performance tests, and thirdly to .evaluate items which 
had already been developed for the test item pool. 

A consensus concerning the curriculum analysis was reached at the 
meetings, and the resulting outline is to serve as a guide in the develop- 
ment of the achievement tests. The final outline was broken into as many 
different areas as the field consultants felt necessary to represent what 
they considered to be independent areas of instruction. Each independent 
area was then subdivided to detail fhe various clients of the curriculum 
contained therein. This breakdown will be used to insure tjiat items for 
the item pool will be representative of all areas of the curriculum. 

At each of the meetings, the concept of performance measurement 
was discussed. In some of the curricula, particularly the two technical 
curricula, doubts were voiced as to the appropriateness of performance 
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measuremept, Instructors in these areps felt that the objectives of these 
curricula were the assimilation, understanding and application of technical 
knowledge with minimum emphasis on the development of manual skills. 

The construction of test items is well underway for each area. As 
instructors develop items, the items are sent to the project staff. The 
items are then edited and evaluated in terms of grammatical correctness, 
format consistency and adherence lo accepted principles of test construction. 
Finally, the items are being reproduced in booklet form for final evaluation 
by the committee of consultants. 

A meeting of all field cpnsultants has been scheduled for Friday 
pight and Saturday, December 2 and 3. At this meeting, all items which 
have been processed as described above will be evaluated by the coipmittee in 
terms of appropriateness of item, correctness of answer, and validity of 
item detractors. All items which pass this evaluation will be placed in 
the item pool from which tests will be compiled* 

As stated in a previous progress report, twelve technical in- 
stitutes and/or community colleges agreed to participate in the initial 
phase of the research project. Since that time, all other institutions 
within the North Carolina Department of Community Colleges which offer one 
or more of the curricula included in the research prqject have been con- 
tacted and asked to participate in the second phase of the research 
project. The second phase of the project will include the administra- 
tion of the preliminary forms of the achievement tests. To this time, 
an additional ten institutions have agreed to participate ip the initial 
testing progrqsif One institution declined to participate on the groupds 
that this is its first year of operation and to this time has been unpble 
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to complete the move into its permanent facilities. Twelve institutions 
consisting primarily of units of technical institutes or community colleges 
have not been heard from to this date. 

In order to have some means of comparing student scores on pre- 
liminary forms of achievement tests developed by this project with an 
instrument of known reliability and validity, it was decided by the staff 
that each achievement battery should contain one or more reference tests, 
either in the area of achievement or aptitude, which possesses a high 
degree of reliability and a known validity. To determine the test or tests 
which would be most appropriate for each curriculum an extensive review of 
aptitude and achievement tests in print is underlay. Tests in six different 
aptitude s re as are being evaluated. These aptitude areas are: numerical 
ability, mechanical ability, verbal reasoning, linguistic ability, spatial 
reasoning, abstract reasoning. Standardized achievement tests in certain 
related areas such as mathematics , physical sciences and English are also 
being evaluated. 

Problems concerning methods and techniques of evaluation of pre- 
liminary forms of the achievement tests have also been considered* At 
this time, it has been decided that various factor analyses will be 
performed between the various subtests in each curriculum. It was 
recognized that the weighting of the factor analyzed variables would be 
very important in the final forms of the achievement tests. Various 
weighting techniques are being studied and compared. 

The problem of data storage and retrieval is being studied 
concurrently with methods and techniques of data evaluation. A study is 
currently underway to determine what types and quantities of equipment 
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such as card punches, sorters, and test scoring equipment ^ill be needed 
to process for computer evaluation the large quantity of data anticipated 
from the first administration of the achievement tests. 

As mentioned in the preceding progress report, the tasK of 
categorizing the various levels of learning and behavioral objectives as 
to number and kind presented a most difficult problem. The Taxonomy of 
Education Objectives, Cognitive Domain by Benjamin Bloom was examined. 

Due to the fact that there is much disagreement and confusion among 
psychologists and educators in applying this system to test items, it was 
decided by the staff to investigate a system of classification of 
educational objectives which would be more suitable to our needs. The sya~ 
tern finally decided upon was a modification of Bloom's Taxonomy and consists 
of four levels of classification. These four levels of classification ar^; 
knowledge, comprehension, and two levels of application. Operational 
definitions of the four levsls of classification have been developed and 
a paper for use by the staff and field consultants has been prepared. 
(Appendix A) . 

In the experimental area, much time was devoted to the tactile 
kinesthetic sensitivity experiment. As stated in the previous progress 
report this study involved the ability to discriminate sm^ll differences 
in weight. All attempts to replicate the study of Fleishman and Rich 
(19$1) proved futile. 

From our preliminary studies, it seems a difference in weight 
as small as two grams cannot be reliably discriminated. Our invesfiga** 
t ions indicated a six gram difference to be the smallest which could be 
reliably discriminated. Using 29 introductory psychology students as 
subjects yielded a 0.67 reliability coefficient on a 24 hour test-retesf . 
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This result was obtained by correlating the total number of incorrect 
choices on the first day with those of the second day. The use of dif- 
ference limens on the same data yielded reliabilities that were not 
significantly different from zero. 

The above study was performed under what we consider to be crude 
circumstances in that the subjects were actually lifting the weights. The 
apparatus which we have designed for testing tactile kinesthetic response 
was described in the previous progress report; this apparatus will be 
ready for use in the near future and will, we feel, overcome some of the 
difficulties of the initial study. Other methods which have more face 
validity in the vocational setting are being designed to test tactile 
kinesthetic response. These will include the use of feeler gajgps, micrqm- 
eters and rods and bushings. 

Much planning and investigation has been done in the ^rea of 
auditory response (Appendix 3) , and its relationship to achievement in 
several of the vocational areas. Test recordings have been made of several 
normal and abnormal sounds produced by an operating automobile engine. The 
equipment used was not of the quality needed for recording for test purposes, 
but we have gained insight and experience. This appears to be a fruitful 
area for further investigation. Suitable quality recording and playback 
equipment has been requisitioned, and plans are underway for further re- 
cording in the areas of automotive mechanics, machine shop, and radio and 
TV repair. 
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k meeting of the advisory committee was he^d on October 25, 1966. 
A progress report was given and problems which we h^ve encountered were 
discussed. The possibility of testing students in states other than North 
Carolina was discussed. The idea was enthusiastically endorsed by the 
advisory committee. We are currently studying the problems to ascertain 
how muph additional testing can be done in other states within the confines 
of our existing budget and with th^ personnel currently assigned to the 
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APPENDIX A 

USE OF OBJECTIVES IN ITEM CONSTRUCTION 
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Once concepts and skills (subject matter content) to be learned have 1 

'1 

been identified, a further breakdown in terms of* intended behaviors must be accom- 

') 

plished in order to complete the specification of instructional objectives. It is I 

•? 

suggested here that the intended behaviors might be classified as knowledge, under- 
standing and application. There are two kinds of application (explained below), \ 

making, in all, four broad classifications of intended behaviors. Ideally, achieve- 
ment tests require all of the intended behaviors, therefore, operational definitions 
of the four classifications, as they pertain to testing, are given in this paper. | 

Items of a test which measure knowledge require the repetition of re- 
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sponses that have been or should have been practiced in learning experiences prior 

i 

i: 

to the time of taking the test. After learning, memory is the major requisite to \ 

i] 

correct performance- \ 

I 

Examples: | 

'■> 

■ jj 

1. An alphabetic character in the Hollerith card code consists of: \ 

a. One numeric punch J 

b. Two numeric punches ] 

c. One zone and one control punch | 

* d. One numeric and one zone punch \ 

2. A control punch in a card is used: } 



a. To define fields 

* b. To identify different types of cards 
c. To control key punching 
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I terns of a test which measure understanding require responses in addition 
to those previously practiced and learned. The additional responses are likely to 

i 

be interpretations, translations, summarizations, analyses, detection of similarities, 
detection of differences, etc. Items at this level do not set tasks which require 
solutions other than explorations in meaning. 

Examples : 

1. In storage, instruction and data: 

a. Must be separated by 100 positions of storage 

b. Should be grouped by length of field 

c. Can occupy the same locations simultaneously 
*d. Should be assigned separate areas 

2. A computer can make logical decisions by: 

a. Comparing 

b. Arithmetic results 

c. Testing signs or characters 
*d. All of the above 

Items of a test which measure application require the use of previously 

learned responses in the solution of problems set by the items. At this 
level, which we shall call application I, the problems are not new having been 
experienced by the testee before to the extent that responses that are necessary 
to find solutions are more or less routine. 

Examples : 
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Answer the following questions based on the above flow-chart. Assumptions are: 
(1) switch is off initially, (2) counter is zero initially, (3) quantity is 
zero initially, (4) file consists of 10 cards. Each card represents a quantity 

1. How many cards will be processed? * 

2. What is the total quantity printed on the first line? ~ 

3. What is the total quantity printed on the second line? ~ 

4o What is the final quantity amount? — 

5o How many cards will be punched? 

6. What is the status of the switch when program halt is 
reached? 
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